Knowledge Reduction and Discovery

Knowledge reduction, includes attribute reduction and value reduction, is an important topic in rough set literature. It is also closely relevant to other fields, such as machine learning and data mining. In this paper, an algorithm called TWI-SQUEEZE is proposed. It is so named because it can find a reduct, or an irreducible attribute subset that maintains certainty of classification, after two scans, which is a bit similar to the process of squeezing water from sponge by pressing on both sides of the sponge. Its soundness and computational complexity are given, which show that it is the fastest algorithm at present. A measure of difference contained in a system, or variety, is brought forward. The quantity character of it is measured by demarcation information, of which algorithm TWI-SQUEEZE can be regarded as an application. Demarcation information measure can also be used as heuristic information to guide the algorithm to find a suboptimal reduct. In this paper, this measure will be compared with Shannon entropy in details. Some basic concepts, such as uncertainty, distinctiveness, variety, similarity, difference and their relationships will be studied. The author also argues the rightness of this measure as a measure of information, which can make it a unified measure for “differentiation”, a concept appeared in cognitive psychology literature. Value reduction is another important aspect of knowledge reduction. It is interesting that using the same algorithm we can execute a complete value reduction efficiently. A complete knowledge reduction, which results in an irreducible table, can therefore be accomplished after four scans of table. The byproducts of reduction are two classifiers of different styles. Traditionally, attribute reduction is regarded as data preprocessing phase in knowledge discovery. The author will show in this paper that, knowledge reduction, especially approximate knowledge reduction which is based on attribute reduction, is a process of data mining. In this paper, various cases and models will be discussed to prove the efficiency and effectiveness of the algorithm. Some topics, such as how to integrate user preference to find a local optimal attribute subset will also be discussed.